Apparatus and method for storing a movie within a movie

ABSTRACT

A hierarchical movie is provided. A hierarchical movie is a movie that contains one or more embedded movies. Embedded movies may themselves contain embedded movies. Each movie contains zero or more media sequences. Within a hierarchical movie, media sequences that should be edited together may be grouped together using embedded movies. The media sequences of a hierarchical movie may be sequenced during playback based on a different time coordinate system than the time coordinate system that governs any embedded movies. This allows a movie to contain both time-based and time-independent media sequences. Also, the relative timing of events in the movie may vary from performance to performance. The hierarchical movie structure allows movies to be used as user interface controls, and even as field-sensitive databases.

This application is a continuation of U.S. patent application Ser. No.09/049,715, filed on Mar. 27, 1998, (now issued as U.S. Pat. No.6,297,830), which is a divisional of U.S. patent application Ser. No.08/570,542, filed on Dec. 11, 1995 (now issued as U.S. Pat. No.5,571,281). This present application hereby claims the benefit of theseearlier filing dates under 35 U.S.C. §120.

FIELD OF THE INVENTION

The present invention relates to a hierarchical movie structure, andmore specifically, to a structure for embedding a movie within anothermovie.

BACKGROUND OF THE INVENTION

A movie generally consists of a set of tracks slaved to a movie clock.In a typical movie, the array of tracks includes a video track and anaudio track. A video track consists of a sequence of samples of videodata. An audio track is a sequence of samples of audio data.

Besides video and audio tracks, movies may also include tracks whichstore other types of information. For example, a movie may also includea text track that contains text for subtitles, a music track separatefrom the main audio track, and a time code track.

FIG. 1 illustrates a typical movie 100. Movie 100 includes a video track122 that includes a sequence of video samples 102, 104, 106, 108 and110. Movie 100 also includes a sound track 124 that includes a sequenceof audio samples 112, 114, 116, 118 and 120. When movie 100 is played,playback of all of the tracks of the movie 100 are synchronized based ona movie clock. For example, at a time T1 of the movie clock, videosample 102 is being displayed and audio sample 112 is being played. At atime T2 of the movie clock, the video sample 104 is being displayed andaudio sample 114 is being played. The audio and video samples that arebeing played at any given time on the movie clock remain the same foreach performance of movie 100.

If one track of a movie is edited, some of the other tracks of the moviemay also have to be edited. Consider, for example, a movie in which acharacter is giving a speech with the national anthem playing in thebackground. If one wishes to delete a portion of the speech from themovie, the corresponding sequence of video must be cut from the videotrack, the corresponding sequence of audio must be cut from the audiotrack, and the corresponding text sequence must be cut from the texttrack. However, to maintain the continuity and integrity of the nationalanthem, the corresponding music track should not be cut.

Flat movie formats provide no mechanism for keeping track ofrelationships between tracks. Because the editing of one track of amovie may require the editing of some but not all of the other tracks inthe movie, movie editing can quickly become a difficult and complextask. Complex editing operations are even more complicated. For example,during an operation in which one movie is created by splicing togethertracks of other movies, it may be virtually impossible to keep track ofwhich tracks should and should not be edited together.

Based on the foregoing, it is desirable to simplify the movie editingprocess. It is further desirable to expand the application of moviesbeyond simple deterministic time-based media playback applications.

SUMMARY OF THE INVENTION

A hierarchical movie is provided. A hierarchical movie is a movie thatcontains one or more embedded movies. Embedded movies may themselvescontain embedded movies. Each movie contains zero or more mediasequences. Within a hierarchical movie, media sequences that should beedited together may be grouped together using embedded movies. The mediasequences of a hierarchical movie may be sequenced during playback basedon a different time coordinate system than the time coordinate systemthat governs any embedded movies. This allows a movie to contain bothtime-based and time-independent media sequences. Also, the relativetiming of events in the movie may vary from performance to performance.The hierarchical movie structure allows movies to be used as userinterface controls, and even as field-sensitive databases.

According to one embodiment of the invention, a hierarchical mediacontainer is provided. The hierarchical media container includes a firstset of media sequences. One media sequence in the first set of mediasequences contains and an embedded media container including a secondset of media sequences.

The hierarchical media container may or may not have the samerelationship to time as the embedded media container. For example, inone embodiment both the hierarchical media container and the embeddedmedia container are time-based media containers. In another embodiment,the hierarchical media container is a time-independent media containerand the embedded media container is a time-based media container. In yetanother embodiment, the hierarchical media container is a time-basedmedia container and the embedded media container is a time-independentmedia container.

According to another aspect of the invention, a method for providing a“control movie” is provided. According to the method, a user may selecta parameter value by interacting with a movie. The method may be used ina computer system that includes a display device.

The method includes a step for providing a media container that includesa media sequence of visual data. The media sequence includes a pluralityof samples. The plurality of samples includes a set of samples. Eachsample in the set is associated with a value. The method also includessteps for determining a current sample from the set of samples,establishing the value associated with the current sample as theparameter value, and displaying on the display device the imagerepresented by the current sample.

The method also includes the steps of receiving input from the userspecifying a sequencing direction, determining a next sample, the nextsample being a sample of the set of samples located in the sequencingdirection relative to the current sample, establishing the valueassociated with the next sample as the parameter value, and displayingon the display device the image represented by the next sample.

According to another aspect of the invention, a method for editing amovie is provided. According to the method, a first plurality of mediasequences of the movie is stored in a first container. A secondplurality of media sequences of the movie is stored in a secondcontainer. One of the first container and the second container areembedded in the other of the first container and the second container.An edit of a media sequence of the second plurality of media sequencesis received from a user. All media sequences of the second plurality ofmedia sequences are automatically edited responsive to receiving theedit.

According to another embodiment of the invention, a method for playing amovie is provided. According to the method, a first plurality of samplesof a first media sequence are sequentially played. The first mediasequence is stored in a first media container. A second plurality ofsamples of a second media sequence is also sequentially played. Thesecond media sequence is stored in a second media container. The secondmedia container is embedded within the first media container.

Optionally, the step of sequentially playing the first plurality ofsamples may be performed responsive to a first time coordinate system,while the step of sequentially playing the second plurality of samplesis performed responsive to a second time coordinate system, where thefirst time coordinate system is different from the second timecoordinate system.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1 illustrates a movie structure used in the prior art;

FIG. 2 illustrates a computer system upon which the present inventionmay be implemented;

FIG. 3a illustrates a hierarchical movie structure according to anembodiment of the invention;

FIG. 3b illustrates a hierarchical movie structure that has more complexhierarchical relationships than that illustrated in FIG. 3a;

FIG. 4 illustrates a movie generated based on data stored in ahierarchical movie structures where a time-based movie is embedded in atime-based movie;

FIG. 5 illustrates a movie generated based on data stored in ahierarchical movie structure where a time-independent movie is embeddedwithin a time-based movie;

FIG. 6 illustrates a time-based movie generated based on a hierarchicalmovie-structure where a time-based movie is embedded in atime-independent movie;

FIG. 7 illustrates a movie sequence for a control movie according to anembodiment of the invention;

FIG. 8 illustrates a movie generated based on a media container with anembedded control movie;

FIG. 9a illustrates a media container for a field-insensitive databasemovie;

FIG. 9b illustrates a database movie generated responsive to the mediacontainer shown in FIG. 9a;

FIG. 10a illustrates a media container for a field-sensitive databasemovie; and

FIG. 10b illustrates a database movie generated responsive to the mediacontainer shown in FIG. 10a.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

A method and apparatus for creating and using a movie within a movie isdescribed. In the following description, for the purposes ofexplanation, numerous specific details are set forth in order to providea thorough understanding of the present invention. It will be apparent,however, to one skilled in the art that the present invention may bepracticed without these specific details. In other instances, well-knownstructures and devices are shown in block diagram form in order to avoidunnecessarily obscuring the present invention.

Referring to FIG. 2, it illustrates a computer system 200 upon which thepreferred embodiment of the present invention can be implemented.Computer system 200 comprises a bus or other communication means 201 forcommunicating information, and a processing means 202 coupled with bus201 for processing information. System 200 further comprises a randomaccess memory (RAM) or other dynamic storage device 204 (referred to asmain memory), coupled to bus 201 for storing information andinstructions to be executed by processor 202. Main memory 204 also maybe used for storing temporary variables or other intermediateinformation during execution of instructions by processor 202. Computersystem 200 also comprises a read only memory (ROM) and/or other staticstorage device 206 coupled to bus 201 for storing static information andinstructions for processor 202.

Furthermore, a data storage device 207 such as a magnetic disk oroptical disk and its corresponding disk drive can be coupled to computersystem 200. Computer system 200 can also be coupled via bus 201 to adisplay device 221, such as a cathode ray tube (CRT), for displayinginformation to a computer user. An alphanumeric input device 222,including alphanumeric and other keys, is typically coupled to bus 201for communicating information and command selections to processor 202.Another type of user input device is cursor control 223, such as amouse, a trackball, or cursor direction keys for communicating directioninformation and command selections to processor 202 and for controllingcursor movement on display device 221. This input device typically hastwo degrees of freedom in two axes, a first axis (e.g., x) and a secondaxis (e.g., y), which allows the device to specify positions in a plane.

Alternatively, other input devices such as a stylus or pen can be usedto interact with the display. A displayed object on a computer screencan be selected by using a stylus or pen to touch the displayed object.The computer detects the selection by implementing a touch sensitivescreen. Similarly, a light pen and a light sensitive screen can be usedfor selecting a displayed object. Such devices may thus detect selectionposition and the selection as a single operation instead of the “pointand click,” as in a system incorporating a mouse or trackball. Stylusand pen based input devices as well as touch and light sensitive screensare well known in the art. Such a system may also lack a keyboard suchas 222 wherein all interface is provided via the stylus as a writinginstrument (like a pen) and the written text is interpreted usingoptical character recognition (OCR) techniques.

Another device which may be coupled to bus 201 is hard copy device 224.Hard copy device 224 may be used for printing instructions, data, orother information on a medium such as paper, film, or similar types ofmedia. Additionally, computer system 200 can be coupled to a device foraudio playback 225 such as a speaker. Further, the device may include aspeaker which is coupled to a digital to analog (D/A) converter forplaying back the digitized sounds. Finally, computer system 200 can be aterminal in a computer network (e.g., a LAN).

In the currently preferred embodiment, computer system 200 is one of theMacintosh® family of personal computers such as the Macintosh® IImanufactured by Apple® Computer, Inc. of Cupertino, Calif. (Apple andMacintosh are registered trademarks of Apple Computer, Inc.). In thecurrently preferred embodiment, the present invention is related to theuse of a computer system 200 to create, store, and play back movies thatcontain other movies.

Terms

In the following discussion, the term “media sequence” refers to aplurality of ordered samples. A video track, for example, is a mediasequence in which each sample contains video data representing an image.Similarly, a sound track is a media sequence in which each samplecontains audio data representing sound.

The term “media container” refers to a data structure that includes zeroor more media sequences. A QuickTime movie is a media container in thatit stores multiple media sequences, such as video tracks, audio tracks,sound tracks, text tracks, etc. QuickTime movies are described in detailin Inside Macintosh:QuickTime by Apple Computer Inc., published byAddison-Wesley Publishing Company (1993). All of the media sequencesthat belong to a media container are sequenced according to a commontime coordinate system.

The terms “embedded movie”, “contained movie”, “embedded mediacontainer” and “contained media container” refer to a media containerthat is a logical component of another movie container. The terms“containing movie” and “containing media container” refer to the mediacontainer of which an embedded movie is a logical component. As shall beexplained below, the media sequences of an embedded movie are notnecessarily sequenced according to the same time coordinate system asthe media sequences that belong to the containing movie.

Significantly, the embedded relationship is a logical relationship, nota physical relationship. Therefore, the data that represents an embeddedmovie is not necessarily located in the same physical file or even onthe same physical storage device as other data “contained in” thecontaining movie. For example, a movie A stored in a file A located on astorage medium A and a movie B that is stored in a file B located on astorage medium B may both be embedded in a movie C that also includestracks D, E and F that are stored in a file G on a storage medium H. Inembodiments where the embedded relationship is not reflected in theactual physical location of data, control data structures are used toreflect the logical relationship between files.

During playback, all media sequences progress from sample to samplebased on the occurrence of an event. When the event is the passage oftime in a time coordinate system, the media sequences are referred to as“time-based” media sequences. A video track is an example of atime-based media sequence. During playback, a sample in a video mediasequence is displayed for a set time interval. After the time intervalexpires, the next sample in the video media sequence is displayed. Thisprocess continues until all of the samples in the video media sequencehave been displayed. The time interval may be modified to speed up orslow down playback, but the playback timing is still driven by thepassage of time.

If the event that causes a media sequence to progress from one sample tothe next is anything other than the passage of time in a time coordinatesystem, the media sequence is referred to as a “time-independent” mediasequence. For example, consider a media sequence in which each samplecontains the text of a page in a novel. During playback, the pagerepresented in a sample should be displayed until the reader hascompleted reading the page. Since reading speeds vary greatly, theplayback mechanism should not display the page associated with the nextsample until the reader indicates a desire to turn the page. Thus, amechanism may be provided to the user through which the user mayinitiate an event to move to the next page. For example, a user mayoperate a mouse or other pointing device to click on a “Turn Page”button to cause the playback mechanism to sequence to the next sample.

A media container is “slaved” to a clock if the clock determines whenthe media sequences that belong to the media container progress from onesample to the next. All of the media sequences in a typical movie areslaved to the same clock (the “movie clock”) to ensure that the mediasequences remain synchronized during playback. A media container is“independent” of a clock if the media sequences within the mediacontainer are sequenced based on an event other than the passage of timeon the clock. For example, a media container is independent of a clockif playback of the media sequences within the media container may beslowed without slowing the clock.

A sequencing direction is the direction in which a media sequence isplayed relative to the order of the samples. Because media sequences are“ordered”, all media sequences have at least two possible sequencingdirections. For the purposes of discussion, these two sequencingdirections will be referred to as “forward” and “backward”. However, itshould be understood that “forward” does not necessarily mean the“normal” or “typical” direction, since some applications may processmedia sequences in one direction, other applications may process mediasequences in the other direction, and yet other applications may processsequences in either or both directions.

The “active interval” of a sample is the time interval during which thesample may be played. For example, the active interval for the firstvideo sample in a 30-frame per second movie is the first {fraction(1/30)} second of playback.

Playback Mechanism

In the preferred embodiment, the mechanism for playing a movie isimplemented through a series of instructions executed on processor 202.Initially, the series of instruction may be stored on storage device207. When the playback mechanism is invoked, the instructions are copiedfrom storage device 207 into memory 204, and then accessed and executedby processor 202.

During execution of the series of instructions, the samples of the mediasequences of a movie are processed by processor 202 responsive to theseries of instructions. Specifically, processor 202 causes the samplesto be “played”. The particular steps for playing a sample depend on thenature of the data within the sample. For example, a sample of videodata is “played” by causing the image represented in the sample to bedisplayed on display device 221. Samples containing audio data areplayed by generating the sound represented in the audio sample. Soundmay be generated, for example, on speaker 225.

Processor 202 sequences through the movie responsive to the series ofinstructions. The series of instructions may cause processor 202 tosequence through the movie responsive to the passage of time and/or theoccurrence of another type of event. An event which causes processor 202to sequence to the next sample in a media sequence may be auser-actuated event, such as the selection of a key on keyboard 222, orthe operation of a user-interface control through actuation of mouse223.

Hierarchical Media Containers

Typical movies include a plurality of time-based media sequences thatare played back based on a common time coordinate system. According toone aspect of the present invention, a media container format isprovided in which media sequences may contain samples that arethemselves media containers. Such media containers may be stored, forexample, on storage device 207.

Referring to FIG. 3a, it illustrates a media container 300 that containsfour media sequences 302, 304, 306 and 311. Media sequences 302, 304 and306 are typical media sequences, such as sound, video or text tracks.Media sequence 311 contains a sample 309 that contains another mediacontainer 308. Media container 308 includes two media sequences 310 and312.

The active interval of sample 309 is between times T1 and T2.Consequently, media container 308 may only be played during times T1 andT2. These times are determined by the mechanism used to sequence mediacontainer 300.

Because one media container 308 is a component of a media sequence 311in another media container 300, a hierarchy exists between the mediacontainers. The data structure that establishes this hierarchy isreferred to herein as a hierarchical media container. As mentionedabove, the “contained in” relationship is logical, not necessarilyphysical. Thus, the data for each of the various media sequences shownin FIG. 3a may be stored in separate files on separate storage devices.

While FIG. 3a illustrates media container 300 with a single embeddedmedia container 308, the hierarchical media container structure allowsmedia containers to have any number of embedded media containers.Specifically, a single media container may contain multiple mediasequences that contain embedded media containers. In addition, a singlemedia sequence may have multiple samples, each of which contains its ownembedded media container. Further, embedded media containers maythemselves contain media sequences that contain embedded mediacontainers. As a result, the structure of media containers may betailored to particular applications.

FIG. 3b illustrates a media container 320 that has a more complicatedhierarchical structure than media container 300 in FIG. 3a. Mediacontainer 320 contains two media sequences 322 and 336, each of whichcontain embedded media containers. Media sequence 322 contains two moviesamples. The first movie sample in media sequence 322 contains a mediacontainer 324, and the second movie sample contains a media container326. Media container 324 contains two media sequences 328 and 330, andmedia container 326 contains two media sequences 332 and 334.

Media sequence 336 contains one movie sample. The movie sample containedin media sequence 336 contains a media container 338 that has threemedia sequences 340, 342 and 344. Media sequence 344 has two moviesamples which contain media container 346 and media container 352respectively. Media container 346 includes media sequences 348 and 350,while media container 352 contains media sequences 354 and 356. Thus,media containers 346 and 352 are embedded in a media container 338 thatis itself embedded in a media container 320.

At least two significant benefits result from the use of hierarchicalmedia containers. First, the logical relationship between related mediasequences may be reflected in the structure of the media containeritself. Second, different media sequences within a single movie may bedriven by different time coordinate systems.

These benefits make it possible to use movies for applications thatpreviously required complex, custom-designed objects. In addition, thetask of editing and combining movies is simplified. Various applicationsmade possible by the ability to embed media containers within mediacontainers shall now be described in greater detail.

Synchronously-Played Embedded Movies

In the simplest example of an embedded movie, all media sequences aretime-based and the time coordinate system that applies to the embeddedmovie is slaved to the time coordinate system that applies to thecontaining movie. For example, assume that media sequences 302, 304,306, 310, 311 and 312 are all time-based media sequences. Because mediasequences 302, 304, 306 and 311 belong to media container 300, mediasequences 302, 304, 306 and 311 will sequence based on a common timecoordinate system during playback. Similarly, because media sequences310 and 312 belong to media container 308, media sequences 310 and 312will sequence based on a common time coordinate system during playback.

In the simplest situation, the time coordinate system used to sequencethe media sequences in media container 300 is the same time coordinatesystem used to sequence the media sequences in media container 308. Inthis situation, media sequences 310 and 312 will be played back as ifthey where simply two more media sequences contained in media container300.

Asynchronous-Played Embedded Movies

The time coordinate system that applies to an embedded movie need not beslaved to the time coordinate system that applies to the containingmovie. Thus, the time coordinate system for media container 300 may be adifferent time coordinate system than that used to sequence mediacontainer 308.

Because media containers embedded in a given media container may havedifferent time coordinate systems than the given media container,various multimedia effects are possible. For example, a user may be ableto speed up or slow down certain aspects of a movie relative to otheraspects of a movie. Consider the situation in which media sequences 302,304 and 306 represent the sound and image of a helicopter 402 as itflies form a point A to a point B, as shown in FIG. 4. Media sequences310 and 312 may represent the sound and image of a car 404 as it travelsfrom a point C to a point D. Because media container 308 is not slavedto the clock of media container 300, playback of the media sequencesassociated with car 404 relative to playback of the media sequences ofhelicopter 402 may vary from performance to performance.

For example, during one performance of media container 300, helicopter402 may begin to move before car 404. During another performance, car404 may begin to move before helicopter 402. Similarly, during oneperformance, helicopter 402 may move faster than car 404. During anotherperformance, car 404 may move faster than helicopter 402. The relativeplayback starting times and playback rates may be based on user input.Thus, users may operate controls to cause helicopter 402 and car 404 torace across the screen, where the outcome of the race is notpredetermined.

Stop Data

When an embedded movie is not slaved to the clock of the containingmovie, it is possible for the playback of the embedded movie to becompleted before the end of the active interval of the sample in whichthe embedded movie is contained. For example, if media container 308 isslaved to a clock that is running twice as fast as the clock associatedwith media container 300, then media container 308 may be played twicebetween T1 and T2. Under some circumstances it is desirable for theembedded movie to play in a continuous loop during the active intervalwith which it is associated. Under other circumstances, it is desirablefor the embedded movie to play once and then stop, even if the activeinterval for the sample in which it is contained has not ended. Underyet other circumstances, it is desirable for the embedded movie to playup to N times, and then stop, where N is some specified number.

Based on the foregoing, one embodiment of the invention allows “stopdata” to be stored for each embedded movie. The stop data specifies acondition under which the playback mechanism is to stop playing theembedded movie. For example, the stop data may indicate that theplayback mechanism is to stop playing an embedded movie after it hasbeen repeated ten times. The playback mechanism reads the stop data andstops playback of the embedded movie when one of the following eventsoccurs: (1) the termination condition specified in the stop data issatisfied, or (2) the active interval associated with the embedded movieends.

Time-Independent Within Time-Based

Hierarchical media containers may be used to mix time-based mediasequences with time-independent media sequences. For example, mediasequences 302, 304 and 306 may be time-based media sequences while mediasequences 310 and 312 are time-independent sequences.

FIG. 5 illustrates one application of a time-independent movie embeddedin a time-based movie. In FIG. 5, media sequences 302, 304 and 306provide the video and sound for a helicopter 502 flying across a screen508. While the helicopter 502 is flying across the screen 508, a usermay browse through a book 504. The text of book 504 may be stored inmedia sequence 310, and the sound of turning pages may be stored inmedia sequence 312. When a user selects the upper portion 506 of a page,the media sequences 310 and 312 are advanced (text of the next page isshown, and the sound of a page turning is generated). The rate at whichthe user turns pages has no affect on the rate at which helicopter 502moves across the screen 508.

Time-Based Within Time-Independent

A media container storing time-based sequences may also be embedded in amedia container that stores time-independent sequences. For example,media sequences 302, 304 and 306 may be time-independent mediasequences, while media sequences 310 and 312 are time-dependentsequences.

Referring to FIG. 6, media sequences 302, 304 and 306 may correspond aseries of static scenes. The user may move from one scene to the next byentering user input to cause media sequences 302, 304 and 306 tosequence to subsequent information samples. One of the static scenes 600may include the image of a television 602. Media sequences 310 and 312may store video and audio that is played on the television 602. The rateat which media sequences 310 and 312 are played is unrelated to the rateat which a user moves from one scene to the next.

Time-Independent Within Time-Independent

A media container storing time-independent sequences may be embedded ina media container that stores time-independent sequences. For example,each sample in the media sequences of the containing media container maycorrespond to a chapter of a book. One of the media sequences of thecontaining media container may contain embedded movies. Each embeddedmovie may contain the text for each page in a chapter of the book. Onebutton may be provided for sequencing the containing movie (to move fromchapter to chapter). A second button may be provided for sequencing theembedded movies (to move from page to page within a chapter).

Editing Operations

With the movie structures of the prior art, editing operations arecomplicated by the fact that edits to some tracks in a movie may requireedits to some but not all other tracks in the movie. Editing operationsmay be simplified by using the hierarchical media container structure ofthe present invention to reflect relationships between media sequences.Specifically, related media sequences may be stored in the same mediacontainer, while unrelated media sequences are assigned to differentcontainers in the hierarchical structure.

Consider the example of a speech made during a performance of thenational anthem. Edits to the video track should be reflected in theaudio track and the subtitle track. However, video edits should notaffect the sound tracks. Using a hierarchical media container structure,the media sequences of the video, audio and subtitle tracks can bestored in a first media container, and the media sequences of the soundtracks can be stored in a second media container. The second mediacontainer can be embedded in the first media container. Editingutilities may then be configured to respond to edits by automaticallyediting all media sequences that belong to the same media container asthe edited media sequence, and to leave all other media sequencesintact.

The process of incorporating clips from many movies into a single movieis also simplified through the use of embedded movies. Consider, forexample, a movie with sixty tracks created by combining tracks fromtwenty different sources. In the prior art movie structure, there is noindication of relationship between tracks. Therefore, the inadvertentdestruction of synchronization between related tracks is difficult toavoid. Using embedded media containers, the relationship between relatedtracks may be maintained.

Specifically, all of the clips from the same source can be stored in thesame embedded media container. Thus, when a media sequence is modified,the editor need only look to the other media sequences in the same mediacontainer to determine whether other media sequences must be modifiedresponsive to the modification. Consequently, embedded containersprovide to editors the ability to maintain logical media sequencegroupings. This ability, in turn, makes complex editing operations moremanageable.

Space Savings

Use of hierarchical media containers may also reduce the size of somemovies. For example, consider a twenty minute movie in which a fourminute musical theme is repeated five times. Using the prior art moviestructure, the movie would contain a sound track covering the fulltwenty minutes of music. Using a hierarchical media container structure,a four minute media sequence could be stored in a first media containerseparate from a second media container that stores the rest of themovie. The first media container may then be embedded in the secondmedia container. Attributes of the first media container may be set sothat playback of the first media container begins with the playback ofthe second media container and continuously repeats until the end ofplayback of the second media container. During playback, the movie willappear and sound the same, but the sound data of the hierarchical mediacontainer will take up approximately one fifth as much storage space asthe twenty minute sound track.

Animated Controls

Many computer applications display “controls” through which a user maydesignate operational parameters. Examples of such controls includescroll bars, check boxes and radio buttons. Controls typically allow auser to select one value from a predetermined range of values. Forexample, a user may designate the value “checked” or the value“unchecked” by interacting with a checkbox. A user may select one of arange of values by interacting with a scroll bar. In general, the morecomplicated the control, the more difficult it is to display and managethe control.

According to one aspect of the present invention, a “control movie” isprovided. A “control movie” is a movie that performs the traditionalfunctions of a control. More specifically, a control movie is a mediacontainer that contains at least one media sequence, where (1) samplesin the media sequence are associated with parameter values and (2) thecurrent sample of the media sequence determines the value of aparameter.

Referring to FIG. 7, a media sequence 700 of a control movie isillustrated. Media sequence 700 is a video media sequence and includesan ordered series of samples 702, 704, 706, 708 and 710 of video data.Each sample of video data in media sequence 700 represents an image.Each sample in media sequence 700 is also associated with a value for aparameter. Specifically, sample 702 is associated with the value “1”,sample 704 is associated with the value “2”, sample 706 is associatedwith the value “3”, sample 708 is associated with the value “4”, sample710 is associated with the value “5” and sample 712 is associated withthe value “6”.

To provide a control interface during the playback of a movie, a mediacontainer that includes media sequence 700 may be embedded into themovie. For example, a movie in which a helicopter 802 flies from a pointE to a point F on a screen 804 is illustrated in FIG. 8. Embedded in themovie is a media container that includes media sequence 700. An image806 corresponding to a sample in media sequence 700 is displayed onscreen 804 during playback of the movie.

Because media sequence 700 is contained in an embedded movie, it doesnot sequence responsive to the time coordinate system that controls thecontaining movie. However, the image 806 generated by the media sequencemay be used to determine a parameter associated with the containingmovie, such as the rate at which the containing movie is played back.

Assume, for example, that the movie associated with helicopter 802 isplayed back at a rate determined by the parameter “speed”. The higherthe value of “speed”, the faster the movie is sequenced during playback.Assume also that the default value for “speed” is “5”.

To use image sequence 700 as a control for the value of “speed”, theimage associated with the default value of “speed” is initiallydisplayed on screen 804. In the present example, the image associatedwith sample 710, which is the sample associated with the value “5”,would be displayed. This image would not change (i.e. media sequence 700would not be sequenced) responsive to the passage of time in the timecoordinate system associated with the containing movie. Rather, someother event, such as user interaction with the image 806, would triggerthe sequencing of media sequence 700.

In one embodiment, media sequence 700 is sequenced responsive to theselection of arrows 810 and 812 on image 806. For example, the selectionof arrow 810 will cause media sequence 700 to sequence “backward”. As aresult, the image 806 will reflect the image associated with sample 708.Also, the value of “speed” will be updated to the value associated withthe currently-displayed sample. The currently-displayed sample will besample 708, which is associated with the parameter value “4”. Thereforethe value of “speed” will be changed to “4”, and the rate of playback ofthe movie associated with helicopter 802 will decrease.

Conversely, the selection of arrow 812 will cause media sequence 700 tosequence “forward”. As a result, the image 806 will reflect the imageassociated with sample 712. Also, the value of “speed” will be updatedto the value associated with the currently-displayed sample. Thecurrently-displayed sample will be sample 712, which is associated withthe value “6”. Therefore the value of “speed” will be changed to “6”,and the rate of playback of the movie associated with helicopter 802will increase.

In the embodiment described above, the sequencing of the control moviecontaining media sequence 700 is performed by selection of arrows 810and 812. This selection may be performed, for example, by operatingmouse 223 to position a cursor over one of arrows 810 or 812 andclicking a button on the mouse 223. Other user actuated sequencingmechanisms may also be used. For example, the sequencing of the controlmovie may alternatively be triggered by the selection of screen regionsoutside of image 806, or by pressing certain keys on keyboard 222.

Control movies have the benefit that once they are created, they mayeasily be embedded in other movies to provide a graphical user interfacefor parameter control. Because control movies are movies, they mayprovide visually sophisticated controls that would otherwise bedifficult to display and manage.

While each sample of media sequence 700 is associated with a parametervalue, other variations are possible. For example, every fifth samplemay be associated with a parameter value. Upon receipt of user inputdesignating a sequence direction, the media sequence 700 may be playedin the designated direction until arriving at the next sample associatedwith a parameter value. Thus, the image 806 displayed responsive to thecontrol movie will appear animated during the parameter changeoperation.

Database Movies

It is possible to search the text track of movies to locate specificwords or word patterns. A movie that consists of only a text track isanalogous to a text document in which each sample of the text trackcorresponds to a different page. FIG. 9a illustrates a media container901 that contains a media sequence 900 in which each sample 902, 904,906, 908 and 910 includes the name and age of a person. A moviecontainer storing media sequence 900 may be played as shown in FIG. 9b.

Referring to FIG. 9b, an image 920 corresponding to the currently-playedsample of media container 901 is displayed on a screen 924. Along withimage 920, a control panel 922 for sequencing media container 901 isalso displayed. Control panel 922 includes control arrows 926 and 928for sequencing media container 901 backward and forward, respectively.In addition, control panel 922 includes a text box 930 into which a usermay enter search terms. Entry of search terms into text box 930initiates a search for terms in the media container 901. If the termsare found, then the media container 901 is sequenced until the samplecontaining the search terms is displayed.

As described above, media container 901 may be used as a rudimentarydatabase. However, the database provided by media container 901 has thedisadvantage that searches are performed on all of the text in the mediacontainer 901. For most database applications, it is desirable to limitsearches to specified fields. For example, one may want to search forsomeone born on the tenth of a month, but not someone who is ten yearsold. If the birth date field cannot be searched separately from the agefield, then a search for “10” will match all instances of “10”,including people who are “10” years old.

The hierarchical media container format described herein may be used tosegregate a movie database into fields. Referring to FIG. 10a, itillustrates a media container 1000 that includes a plurality of mediasequences 1016, 1017, 1019, and 1021. Media sequences 1017, 1019 and1021 respectively contain media containers 1004, 1006 and 1008. Each ofthe embedded media containers 1004, 1006 and 1008 includes a text mediasequence 1010, 1012 and 1014, respectively.

Media sequence 1016 is a media sequence that stores the names ofdatabase fields, such as “Name”, “Age” and “Birthday”. Each sample ofmedia sequence 1010 stores text that indicates the name of anindividual. Each sample of media sequence 1012 stores text thatindicates the age of an individual. Each sample of media sequence 1014stores text that indicates the birthday of an individual. Mediasequences 1010, 1012 and 1014 are ordered such that at any givensequence location, all three media sequences represent data from thesame individual. For example, samples 1018, 1020 and 1022, which are alllocated at the first sequence position, respectively store the name, ageand birthday of the same individual.

Referring to FIG. 10b, media container 1000 may be “played” to displayan image 1050 on a screen 1052. The image 1050 includes a region 1054 inwhich the current sample of media sequence 1016 is displayed, a region1056 in which the current sample of media sequence 1010 is displayed, aregion 1058 in which the current sample of media sequence 1012 isdisplayed, and a region 1060 in which the current sample of mediasequence 1014 is displayed.

Screen 1052 also contains a control panel 1062 that contains controlarrows 1064 and 1066 analogous to control arrows 926 and 928 of FIG. 9b,and a text box 1068 analogous to text box 930 of FIG. 9b. Control panel1062 also contains controls 1070, 1072 and 1074 that allow a user tochoose one or more of the available fields. Because the data for eachfield is contained in a separate media container, searches may beperformed on a field-by-field basis. For example, if a user selectscontrol 1072 and enters “10”into text box 1068, the search is limited tothe contents of media container 1006. If a match is found, mediacontainer 1000 is sequenced until the sample of sequence 1012 in whichthe match occurred is displayed.

As is evident by the foregoing, the hierarchical movie structuredescribed herein allows movies to be applied to applications that havepreviously required complex, customized programming. A singlehierarchical movie, in the form of a media container that containsembedded media containers, can contain an entire multimedia application.Further, all or some of the embedded movies may be time-independent.Thus, the timing of one segment or aspect of the resulting movie mayvary from performance to performance relative to other segments oraspects of the movie. In addition, the ability to group related mediasequences into containers simplifies the movie editing process.

While specific embodiments of the present invention have been described,various modifications and substitutions will become apparent to oneskilled in the art by this disclosure. Such modifications andsubstitutions are within the scope of the present invention, and areintended to be covered by the following claims.

What is claimed is:
 1. A method of displaying graphics on a displaydevice of a digital processing system comprising: displaying on thedisplay device a first series of samples of a first media sequence heldin a hierarchial media container; and displaying on the display device asecond series of samples of a second media sequence held in an embeddedmedia container in the hierarchial media container, playback of thefirst series of samples being adjustable without adjusting playback ofthe second series of samples.
 2. The method of claim 1 wherein starttime of the first series of samples is adjustable without adjustingstart time of playback of the second series of samples.
 3. The method ofclaim 1 wherein playback speed of the first series of samples isadjustable without adjusting playback speed of the second series ofsamples.
 4. The method of claim 1 wherein playback of the first seriesof samples is time-dependent.
 5. The method of claim 1 wherein playbackof the second series of samples is time-dependent.
 6. The method ofclaim 1 wherein playback speed of the second series of samples isadjustable without adjusting playback speed of the first series ofsamples.
 7. The method of claim 6 wherein playback of the second seriesof samples is time independent.
 8. The method of claim 1 whereinplayback of the second series of samples is time independent.
 9. Themethod of claim 1 wherein playback of one of the series of samplesrepeats while playback of the other series of samples plays withoutrepeating.
 10. The method of claim 9 wherein playback of the secondseries of samples repeats.
 11. The method of claim 1, wherein saidembedded media container is capable of holding a plurality of mediatypes.
 12. A computer readable medium having stored thereon computerprogram instructions to display graphics on a display device of adigital processing system, the instructions when executed by the digitalprocessing system causing the system to perform a method comprising:displaying on the display device a first series of samples of a firstmedia sequence held in a hierarchial media container; and displaying onthe display device a second series of samples of a second media sequenceheld in an embedded media container in the hierarchial media container,playback of the first series of samples being adjustable withoutadjusting playback of the second series of samples.
 13. The computerreadable medium of claim 12 wherein start time of the first series ofsamples is adjustable without adjusting start time of playback of thesecond series of samples.
 14. The computer readable medium of claim 12wherein playback speed of the first series of samples is adjustablewithout adjusting playback speed of the second series of samples. 15.The computer readable medium of claim 12 wherein playback of the firstseries of samples is time-dependent.
 16. The computer readable medium ofclaim 12 wherein playback of the second series of samples istime-dependent.
 17. The computer readable medium of claim 12 whereinplayback speed of the second series of samples is adjustable withoutadjusting playback speed of the first series of samples.
 18. Thecomputer readable medium of claim 17 wherein playback of the secondseries of samples is time independent.
 19. The computer readable mediumof claim 12 wherein playback of the second series of samples is timeindependent.
 20. The computer readable medium of claim 12 whereinplayback of one of the series of samples repeats while playback of theother series of samples plays without repeating.
 21. The computerreadable medium of claim 20 wherein playback of the second series ofsamples repeats.
 22. The computer readable medium of claim 12, whereinsaid embedded media container is capable of holding a plurality of mediatypes.
 23. An apparatus for displaying graphics comprising: means fordisplaying on a display device a first series of samples of a firstmedia sequence held in a hierarchial media container; and means fordisplaying on the display device a second series of samples of a secondmedia sequence held in an embedded media container in the hierarchialmedia container, playback of the first series of samples beingadjustable without adjusting playback of the second series of samples.24. The apparatus of claim 23 wherein start time of the first series ofsamples is adjustable without adjusting start time of playback of thesecond series of samples.
 25. The apparatus of claim 23 wherein playbackspeed of the first series of samples is adjustable without adjustingplayback speed of the second series of samples.
 26. The apparatus ofclaim 23 wherein playback of the first series of samples istime-dependent.
 27. The apparatus of claim 23 wherein playback of thesecond series of samples is time-dependent.
 28. The apparatus of claim23 wherein playback of the second series of samples is adjustablewithout adjusting playback speed of the first series of samples.
 29. Theapparatus of claim 27 wherein playback of the second series of samplesis time independent.
 30. The apparatus of claim 23 wherein playback ofthe second series of samples is time independent.
 31. The apparatus ofclaim 23 wherein playback of one of the series of samples repeats whileplayback of the other series of samples plays without repeating.
 32. Theapparatus of claim 31 wherein playback of the second series of samplesrepeats.
 33. The apparatus of claim 23, wherein said embedded mediacontainer is capable of holding a plurality of media types.